A problem is considered ill-posed if there is insufficient information available to achieve a unique solution. In general, a deconvolution problem can be termed ill-posed if there are any singularities in the transfer function or if the observation data to be deconvolved are corrupted by noise. This is often the case in signal processing related to spectroscopy and imperfect imaging systems. The goal of the signal processing is to reconstruct the original object, i.e., to deconvolve the observations and to cancel noise. In other words, the problem is to invert equation (1), below, to find the unknown object O from the known convolution function T and known observation data I: EQU I=O*T (1)
where * represents the convolution operation.
Numerous methods exist for such a deconvolution which can readily be implemented on a digital computer. A simple example is that of Fourier deconvolution. As convolution can be expressed as the product of the transforms of the object and the transfer function, the transform of the object can be found by dividing the transform of the observations by that of the transfer function. However, this approach breaks down when there is noise present in the observations or the transfer function transform has zeros. In such cases the deconvolution can be solved by using a "regularizer" which is a function of successive estimates made of the object. The technique is iterative in nature as the successive estimates O are convolved and compared to the observation, I. One then minimizes EQU .vertline.O*T-I.vertline..sup.2 +.vertline.G(O).vertline.
where G(O) is the regularizer. The regularizer is best chosen to reflect an important characteristic of the specific problem at hand. The problem, however, remains as to the best choice for regularizer in the absence of specific prior knowledge of the object being reconstructed.
The technique of maximum entropy is often described as the preferred method of recreating a positive distribution, i.e., containing only nonnegative values, with well-defined moments from incomplete data. Maximum entropy has been demonstrated to be extremely powerful in several fields such as optical image enhancement, deconvolution, spectral analysis and diffraction tomography. The essence of the maximum entropy method is to maximize the entropy of the reconstructed distribution subject to satisfying constraints on the distribution. These constraints are often defined by a set of observations such as, for example, a moment (e.g., average) or convolution (e.g., blurred image) of the true distribution. Thus entropy is a meaningful choice for a regularizer when the only specific knowledge about the object being reconstructed is positivity. Furthermore, the regularized reconstructions have error term distributions that are of the exponential family. That is, they have well-defined means and variances, which is what one would expect from a real physical system. In contrast least squares estimates do not assure this.
Maximum entropy methods are computationally intensive and require at least a minicomputer and the necessary software. As a result it is difficult to achieve a maximum entropy deconvolution, for example, in real time which would be of great use in many applications. This type of problem would appear to be suited to computation in a multiply connected or "neural" electronic net. Such a net can be designed so that its operation is characterized by a stability (Lyapunov) function which is a well-defined function of the net parameters (i.e., inputs, outputs, interconnects etc.) (An example of such a net is shown in U.S. Pat. No. 4,849,925 to Peckerar and Marrian, the disclosure of which is incorporated herein by reference. This patent discloses a net of the Tank-Hopfield kind modified to use entropy as a regularizer.) The output from such a net evolves with time until a minimum in its Lyapunov function is reached. Here two nets are interconnected: a signal net (also called a variable plane or variable net) representing the solution which receives input from a constraint net when the solution breaks any of a set of constraints. The combined nets give a solution which minimizes a specific cost function subject to the set of constraints being satisfied.
This specification describes a method suitable for implementation in a multiply connected net which gives maximum entropy solutions to ill-posed problems.